Comparing extremal and thermal Explorations of Energy Landscapes 
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Using a non-thermal local search, called Extremal Optimization (EO), in conjunction with a 
recently developed scheme for classifying the valley structure of complex systems, we analyze a short- 
range spin glass. In comparison with earlier studies using a thermal algorithm with detailed balance, 
we determine which features of the landscape are algorithm dependent and which are inherently 
geometrical. Apparently a characteristic for any local search in complex energy landscapes, the 
time series of successive energy records found by EO also is characterized approximately by a 
log-Poisson statistics. Differences in the results provide additional insights into the performance 
of EO. In contrast with a thermal search, the extremal search visits dramatically higher energies 
while returning to more widely separated low-energy configurations. Two important properties 
of the energy landscape are independent of either algorithm: first, to find lower energy records, 
progressively higher energy barriers need to be overcome. Second, the Hamming distance between 
two consecutive low-energy records is linearly related to the height of the intervening barrier. 
PACS number(s): 05.40.-a, 75.10.Nr, 02.60. Pn. 



I. INTRODUCTION 

The exploration of complex energy landscapes poses 
a series of problems of wide interest. Their multi- modal 
geometry is on one side challenging for optimization algo- 
rithms attempting to find the global minimum 1], while 
on the other side it provides a framework to model slow 
relaxation dynamics in nature j^]. Typically, in a physi- 
cal (thermal) exploration of such relaxation phenomena, 
the static (geometric) and dynamic (algorithmic) aspects 
are intertwined. To disentangle the inherently geometric 
features, we apply a decidedly non-thermal optimization 
algorithm, called Extremal Optimization (EO) djy, U, 
to explore the energy landscape of a spin glass whose 
structure has been studied recently with a thermal algo- 
rithm ^. Furthermore, the comparison with the thermal 
algorithm highlights distinct performance features of the 
EO algorithm. 

Focusing on the temporal succession of energy values 
of record magnitude, we present a set of measures which 
characterizes the difficulty of local searches and the com- 
plexity of the landscape. There are at least two geo- 
metrical features of the spin glass landscape which are 
robust. For one, progressively higher energy states have 
to be surmounted in order to reach ever lower energy 
records. Second, a linear relation emerges between the 
Hamming distance of consecutive low-energy record con- 
figurations and the height of the highest intervening en- 
ergy state. Such a relation has previously been found 
by other authors using different local search methods, 
as well as for other models |a, 0, Q- Yet, in stark con- 
trast to the thermal method, EO reaches a rapidly (ex- 
ponentially) growing succession of high-energy records. 



projecting the search through configuration space by an 
exponentially growing Hamming distance between con- 
secutive low-energy records. 

In the following section, we describe how the landscape 
features explored by the dynamics can be assessed on 
the basis of a time series of suitably defined 'valleys'. In 
Sec. mil we review the Extremal Optimization heuristic 
used here to produce such a time series for a non-thermal 
relaxation process. The main results of our numerical 
studies are discussed in Sec. lIVI and in Sec. we present 
our conclusions. 



II. ENERGY VALLEYS IN COMPLEX 
LANDSCAPES 



The idea that progressively deeper, i.e. thermally 
more stable valleys are explored by the thermal dy- 
namics of complex system is well-established and ac- 
counts for many imp ortant features of aging dynam- 
ics |i[ll[ll[ll[ll[ll[ll[ll[ll[l3. Qualitatively 
speaking, a valley would be a set of configurations suffi- 
ciently close to a local energy minimum. These config- 
urations are repeatedly visited by the dynamics before 
the valley is "exited" and altogether different regions are 
explored. Ideally, in thermal systems the states within 
a valley would be visited with frequencies given by the 
respective Boltzmann weights, i.e. a local thermal equi- 
librium state is established before the valley is left. 

The standard way of giving these concepts an opera- 
tional meaning is to subject the dynamics to repeated 
thermal quenches, each leading to a local energy mini- 
mum or Intrinsic State (IS). The set of IS thus obtained 
partitions configuration space into basins of attraction 

with respect to thermal quenches, and each basin can be 

considered as a valley. A different approach 5], which 

* Electronic address: F^.physics.emory.edu/facuIty/boettcher| we pre sently generahze to non-thermal algorithms such 

t Electronic address: IPermanent Address: Fysisk Institut, SDU, Odense^SKp , USeS a time series of unperturbed energy data. It 










A 


' 


, 


' 




-550 


A 

h , 1 1 , 






L=7, T=0.7 


. 




-555 


1 


w 


j|JJ,|,Ia a ; 


>> 


-560 








Hi I , 1 ||1,U 


j.iLiiii 


CD 
C 
LU 


-565 






w^ 


lll'ii UlliliiijIiliillillMBi 




-570 
























V 


^^m 




-575 












■ 




-580 












,n 



time 



1000 10000 

Updates 



100000 



FIG. 1: The definition of a valley is illustrated with a fictitious 
series of energy values. A search produces a time sequence of 
energy (E) and barrier (B) records, where each "E" labels the 
lowest energy seen so far, and "B" refers to the highest barrier 
{relative to the most recent "E") reached up to that time. As 
explained in the text, in the end, only the highest barriers 
and the lowest energy records in each subsequence of "E"s and 
"B"s is kept, and the intermediate values are stricken from the 
record to give a strictly alternating sequence "EBEBE...". In 
particular, any two subsequent "B"s demarcate (entrance to 
and exit from) valleys, here separated by vertical dashed lines. 



defines valleys on the basis of the lowest energy visited 
so far and is motivated by the fact that, for thermal dy- 
namics, the dynamics is recurrent and equilibrium-like as 
long as this lowest energy state does not change. 

As the details can be found in Ref. [a|, only a brief 
account will be given for completeness (see Fig. 0: We 
keep track of the current lowest energy value E encoun- 
tered up to time t and always measure the energy of the 
current state as the difference from this particular value. 
We furthermore keep track of the highest energy bar- 
rier B visited, of the times at which both low and high 
records occur, and of the corresponding configurations. 
A trajectory is thus mapped into a symbolic sequence 
such as . . . EEEBBBBBEEEEEEBBEEE . . ., where 
the number of symbols in each subsequence of contiguous 
E or B values is larger or equal to one. 

Consider first any subsequence of £"s: since a trajec- 
tory 'sliding' downhill will produce such a sequence, all 
but the last E correspond to transient states. By con- 
trast, the last E record may stand for a long time, i.e. 
at least until a record high energy value, the first subse- 
quent i?, is encountered. This makes it a good proxy for 
the lowest energy value in the 'current' valley. Similarly, 
while a trajectory explores high energy states it will likely 
visit several closely spaced energy maxima, producing a 
subsequence of B values. The last B value remains a 
record for long time, and is chosen to mark the passage 
from one valley to the next. In summary, all finite subse- 
quences are pruned to their last element, producing a se- 
quence . . . BEBEBE . . . where each triplet BEB marks 
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FIG. 2: Plot of a typical run with a thermal search (above) 
and an extremal search (below) for a d = 3 Gaussian spin 
glass of size L = 7. The fiuctuating line marks the sequence 
of energies visited by the search. In terms of the definition in 
the text and Fig. Q energy records (E) are marked by down- 
triangles, barrier records (B) by up-triangles. The barrier 
records also demarcate the beginning and the end of a valley, 
so each time interval between two consecutive vertical lines 
constitutes a valley. Counting valleys starts (with nv ~ 0) 
for updates > A'^ (where N = 7^ — 343 here) to avoid early 
transient behavior. While the absolute energy scale between 
both searches is not significant here (two distinct bond real- 
izations were used) , the difference in range and shape of the 
fiuctuations is remarkable. 



a valley, see Fig. ^ The Bs mark the enter and exit 
events and the the E marks the lowest energy of the val- 
ley. The last sequence of i?'s or B's encountered in the 
simulation is not terminated and is discarded. 

We note that this scheme produces trivial results (very 
few or no valleys, only infinitesimally increasing barriers, 
nearly same configurations for different E 's) in cases 
where there is only one global minimum, or when the 
ground state can be chosen as starting point. Consider 
e.g. the case of a Metropolis random walk in a discrete 
set of energy value with a single minimum at energy zero. 
The downward drift implies that this minimum would 



most likely be reached without intervening barrier max- 
ima, and the scheme will produce no output. In cases 
where a few 'false positives' are produced, i.e. valleys 
with no physical counterparts, these will closely resemble 
each other in terms of their configurations, energies etc. 
and will be immediately identified as such in the subse- 
quent analysis. In any case, the time needed to reach the 
global minimum depends linearly, or in the lack of a bias, 
quadratically on the initial energy. The situation is com- 
pletely different in complex energy landscapes, where, 
for thermal dynamics Q , and as shown below, for EO as 
well, the scheme gives a succinct but highly informative 
description of the dynamics: new valleys are accessed 
on a logarithmic time scale, and there is a systematic 
variation of their properties with the 'valley index'. We 
finally note that subsequent valley are defined on coars- 
ening energy and time scales. Hence, each valley can be 
expected to contain many valleys of the previous kinds 
as sub-features. An illustration of the valley structure in 
a thermal and an extremal search is given in Fig. |21 

From an optimization point of view, the E values in 
the series represent the best the algorithm can do on a 
given time scale. The B values act as energy barriers for 
a thermal type of algorithm, but not for an algorithm of 
the EO type, where energy differences have less dynam- 
ical significance. Nevertheless, if, as we expect, a geo- 
metric relationship links the Hamming distance between 
two "sufficiently low" minima and the height of the in- 
tervening barrier, the link should appear irrespective of 
the algorithm chosen. As shown below, the expectation 
is borne out by our simulations. 



larity of these physical systems to many practical combi- 
natorial optimization problems in computer science pro- 
vides additional incentive for a broad-based investigation 
of local search methods and their ability to exploit the 
landscape geometry. EO in particular has proved to be a 
competitive heuristic to determine low-energy configura- 
tion for some of the hardest combinatorial optimization 
problems known, including graph bipartitioning 21], col- 
oring 0, I23I , and also spin glass problems 0| . We can 
hope that a more detailed view at the interplay of heuris- 
tic search and landscape geometry will lead to improve- 
ments in the quality of the results found as well as in the 
speed of convergence. 

The extremal optimization algorithm, r-EO, which we 
employ in this pap er, has been discussed previously in 
I4J , and in |24L |25| with regard to the setting of its sole 
free parameter, r. At each instant during the search of 
a particular instance, t-EO assigns to each spin Xi in 
the configuration its contribution to the total energy as 
"fitness," 



^i ~ n^i /_^ 'JiJ ^j n /_^ \'Ji,j\ J 



(1) 
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where the summation extends over all neighboring spins 
Xj of Xi. Note that the second term on the right corre- 
sponds to the (absolute) weight attributable to that spin; 
it ensures that for each variable its optimal fitness is zero, 
irrespective of its overall weight. Accordingly, 
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III. EXTREMAL OPTIMIZATION 

An energy record statistic as described in the previous 
section can be generated by a variety of dynamical rules. 
Physically most relevant are those which evolve according 
to a thermal process that preserves detailed balance. The 
record statistics of a thermal process has been extensively 
studied previously for a number of different systems p| , 
including the Edwards-Anderson model with Gaussian 
couplings on a cubic lattice. A priori, it is not obvious 
which of the properties of this process can be attributed 
to the dynamic update rule, and which are inherently 
properties of the system. 

A significant alteration of the update dynamics may 
in turn elucidate the origin of certain properties. To 
this end, we consider the Extremal Optimization (EO) 
heuristic ^,^ as a decidedly different, athermal, update 
rule to ex plor e the system. EO, like Simulated Annealing 
(SA) |19l I20II , attempts to advance toward lower energy 
values via a local search of the landscape. Unlike SA, EO 
is modeled after driven dissipative processes, intention- 
ally pushing the dynamics away from local equilibrium 
and detailed balance. 

While a distinction between static landscape and dy- 
namic properties is desirable in its own right, the simi- 



i. e. the sum of all fitnesses tallies the total energy, aside 
from a trivial offset. 

During a search with r-EO, we rank all Xi according 
to fitness Xi, i.e., we find a permutation H of the variable 
labels i with 
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The variable Xj with the worst burden on the total energy 
is of rank 1, j — n(l), and the best variable is of rank 
n. Consider a scale-free probability distribution over the 
ranks k, 



Pk oc k' 



l< k <n, 



(4) 



for a fixed value of r. At each update, select a rank k 
according to Pk. The spin Xj with j = H(fc) is forced 
unconditionally to change state. For r > this selec- 
tion process ensures a certain preference in fixing the 
state of spin variables which put a higher burden on the 
total energy. In particular, it has been found that in- 
termediate choices for the value of r in Eq. Q, with 
r — 1 ro l/ln(n) |2J|, often lead to the best results for a 
given runtime of the algorithm. Values of t much larger 
or smaller than these quickly produce too confined or 
too random searches, as we will see. This issue has been 
explored also in Ref. |26l |. 



The definition of fitness, which generally permits vari- 
ations that can significantly impact performance ^, as 
given in Eq. (^ is purely a measure of "badness" in each 
variable. The constant offset in each Ai in Eq. (Q, con- 
sisting of the absolute weight of all attached bonds, en- 
sures that perfectly well adapted spins, i. e. those bor- 
dering on bonds that are all satisfied, have zero fitness, 
irrespective of their overall weight. Spins bordering on 
unsatisfied bonds get penalized according to their bur- 
den on the total energy. Considering that the absolute 
sum of all weights associated with spins is distributed 
unevenly, heavy-weighted spins (and bonds) are satisfied 
with a higher priority. At later stages of the search, the 
overwhelming number of weaker bonds are attended to. 



IV. NUMERICAL RESULTS 

In this section, we present the results of extensive nu- 
merical investigations of the r-EO search for the d — 3 
Edwards-Anderson spin glass. The procedure, as out- 
lined in Sec. In] follows closely that of Ref. i^i]. There, 
a rejection free implementation of the Metropolis algo- 
rithm, the Waiting Time Method (WTM) W^ , was used 
to study the thermal dynamics of this spin glass at low 
temperatures starting from a hard quench. In the pro- 
cess, many salient features of the record statistics have 
been measured, which we will focus on also in this study. 

We run the r-EO search for a large number of instances 
of varying lattice size to explore the finite-size scaling 
properties of the observables with sufficient accuracy. In 
particular, we have used from 100,000 bond realizations 
for i = 8 to 1,000 such instances for L — 20 and per- 
formed 3 runs each. As in the thermal process, each run 
starts from random initial conditions in the spin configu- 
ration. A form of local "equilibration" is reached within a 
few EO-sweeps of the system, during which each spin gets 
to arrange itself with the local field imposed by its neigh- 
bors. After one sweep, we start sampling low-energy and 
barrier records through a sequence of valleys. In particu- 
lar, we run the EO algorithm on each instance for O(iV^) 
update steps, where N = L^, or about lOA^ sweeps of the 
system. Here, a "sweep" refers to N EO-update steps, 
which are stochastic and do not imply that each variable 
is updated exactly once. Note that the valley index is 
gauged to be ny = after the first (f = 1) sweep of the 
system. 



A. Varying r 

First, we have studied the dependence of EO's perfor- 
mance on the parameter t. To this end, we have con- 
ducted about 10,000 runs of EO on random instances at 
fixed system size L = 16 for t = 0.2, 0.7, 1.2, 1.7, and 
2.2. The behavior of EO has previously been show to 
be very sensitive to this parameter. Overall, we notice 
that, indeed, the results of the record statistics are as 
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FIG. 3: Plot of the valleys found by EO on a logarithmic 
timescale for a system of L = 16 but with values of r = 0.2, 
0.7, 1.2, 1.7, and 2.2. The exploration of new valleys occurs 
at a much faster rate for intermediate values of r, until at 
later times saturation due to finite size effects permits runs 
with higher t to catch up. 



well strongly r-dependent. This dependence is often not 
monotone in r. 

Most significantly, as Fig. O shows, the most valleys, 
and the best energy records, are obtained at intermedi- 
ate values of T, i. e. those valleys are found on much 
shorter timescales than for r values that are too large 
or too small. Since there is a rapid gain in new val- 
leys for T = 1.2, saturation effects due to finite system 
size set in faster, a trend to which higher values of r 
eventually catch up. Overall, there is a more pronounced 
variation with the parameter r than in the corresponding 
data for a thermal search (see Fig. 1 in Ref. Q). From an 
optimization standpoint, r-EO progresses through these 
valleys with about ten times fewer sweeps than a fixed- 
temperature search. 

The trend that an intermediate value of r provides 
the favorable search results is also refiected in the energy 
records found in those valleys, as Fig.^shows. Thus, for 
an "optimal" value of r near unity, EO finds new valleys 
faster and the energy states accessed within those valleys 
are lower. In fact, the exploration for larger values of t 
proves qualitatively similar, just on a slower timescale. 
The behavior for r < 1 is quite distinct, more akin to a 
high-temperature thermal diffusion. These observations 
are consistent with the phase transition at r = 1 in the 
search dynamics of r-EO found for a model problem in 
Ref. "2!. 

At low values of r the search is too random, and what- 
ever valleys are found are not explored with sufficient 
"greed." This randomness expresses itself also in very 
high energy states accessed in between valleys, and the 
total decorrelation between successive valleys as illus- 
trated by the large Hamming distances between the con- 
secutive records in energy. (Hamming distance refers to 
the number of spin flips that separates any two configu- 
rations.) These features are displayed in Figs. [S] and El 
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FIG. 4: Plot of the energy records found by EO within a 
given vaUey as a function of the valley index nv ■ Not only do 
we find new valleys faster at intermediate r ~ 1.2 (as Fig. |21 
shows), but also are lower energy records found within, even 
at equalized valley index ny. (For n — > oo, average ground 
state energies are « —1.70 l2al .'l Note the dramatic shift 
between r = 0.7 and r — 1.2. (Data for r — 0.2 is even 
further above that for r = 0.7 and has been dropped here.) 



in which the barrier records and Hamming distances of 
consecutive energy records are plotted for each vaUey in- 
dex. These properties are quite monotone in r as can be 
expected from the nature of the optimal EO search as a 
compromise between too random and too greedy behav- 
ior [2J,|23- Hence, on the greedy side, for larger r, EO 
spends a long time in each valley, while reaching down to 
very low-energy states within each valley, before escaping 
through an equally low barrier, which only provides ac- 
cess to a new valley with states highly correlated (small 
Hamming distance) with those in the previous one. Con- 
versely, for smaller r, the search approaches a random 
walk through the configuration space that is unlikely to 
reach down to very low-energy states. Any memory of 
fit variables in the list in Eq. Q is short-lived, and the 
search trajectory quickly decorrelates such that the Ham- 
ming distances between consecutive barrier records soon 
saturate at the system size {n/2), see Fig. El 

Most surprising is the rapid increase of both, barrier 
heights and Hamming distances, which scale exponen- 
tially with the valley index for sufhciently large values of 
T. This is in marked difference with the behavior of the 
thermal relaxation, where both grow about linearly with 
the index (see Figs. 2 and 3 in Ref. Q)- In contrast, near 
T = (which rigorously corresponds to T = oo) we re- 
cover the linear scaling of the barrier heights with valley 
index observed for a thermal search, as the upper plot in 
Fig. |S1 demonstrates, again a hint of the phase transition 
at r = 1. In general, barrier heights and Hamming dis- 
tances vary significantly for both, extremal and thermal 
exploration, with their respective parameters. 

Despite this difference between extremal and thermal 
exploration, in both cases the scaling of the Hamming 
distance itself with the barrier, as shown in Fig. \7\ is 
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FIG. 5: Plot of the highest energy level (or barrier) accessed 
between two successive low-energy records as a function of 
valley index ny . (Note that the height is measured relative to 
the most recent low-energy record.) Unlike in Figs. |3 and 0] 
the r-dependence here is monotone. Yet, for r < 1 (upper 
Figure) the barrier height varies only linearly with the valley 
index, similar to a thermal search, while for r > 1 (lower 
Figure) barrier heights rise exponentially. 
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FIG. 6: Plot of the Hamming distance between successive 
low-energy records as a function of valley index ny . As in 
Fig. El the r-dependence here is monotone, showing that the 
search gets increasingly narrow for increasing r. For small t, 
the Hamming distance quickly saturates at the system size 
(n/2). 
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FIG. 7: Plot of the Hamining distance between successive 
low-energy records as a function of the intervening barrier 
height, obtained from Figs.|^and|S|by eliminating the valley 
index dependence between them. The relation for each value 
of r appears to be in fact linear, even for t < 1 before the 
Hamming distances saturate. Linearity is exemplified by the 
dashed line of slope 1; the log-log scale was merely chosen for 
better visibility. 



quite linear aside from finite-size effects. Hence, this is 
all but the first indication of a quantity attributable the 
landscape geometry itself. The purely geometric origin 
of this feature is further emphasized by the fact that the 
data for all t very nearly collapses onto a single line, 
showing only a weak r-dependence in the slope. While it 
is not too surprising to obtain such a linear relation from 
the ratio of two linear relations for the thermal search in 
Ref. [3, in turn, it is outright amazing to extract a sim- 
ple linear relation from the ratio of two exponentials in 
the case of EO here. A linear relationship between Ham- 
ming distance and barrier records was obtained long ago 
for the SK- model 0, 01 1 and was inferred experimentally 
from thcrmo-remanent magnetization data |l(]|. We also 
note that the largest Hamming distance achievable for 
a given barrier, as opposed to the typical one, grows 
exponentially with the barrier in both 2d and 3d spin- 
glasses pa |. 

A qualitative snapshot of the difference between con- 
secutive record configurations is provided in Fig|5| There 
we plot the clusters of overturned spins between two en- 
ergy records at ny = 9 and ny = 10 for some random 
instance of L = 16 and various r. As can be expected 
from Fig.|Hl these records differ by fewer spins for increas- 
ing T. Hence, for low r, the interfaces between Sipped 
and unfiipped spins percolates and is rather indistinct, 
for T > 1 isolated individual clusters become discernible. 

Further quantitative insight into the scope of the 
search can be obtained by looking at the overlap between 
earlier-found configurations and those arrived at for later 
times of the search. To this end, we store the optimal 
configuration found after one sweep in valley ny — 
and measure its overlap with later record energy config- 
urations in valley ny = k. This allows us to define a 
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FIG. 8: Snapshot of the clusters of spins which have changed 
between the configurations of two consecutive energy records 
at nv = 9 and 10 found by r-EO with r = 0.2, . . . , 2.2, for 
some instance of L = 16. Patches cut across bonds along 
which the relative state of adjacent spins has changed. All 
surfaces are actually closed and only appear to be open due 
to the periodic boundary conditions. For small r, all spins 
appear to have flipped many times, and it is impossible to 
tell "inside" from "outside," while for larger r a large number 
of spins clearly have remained "frozen" . It appears that for 
the optimal r = 1.2 flipped spins just about percolate. 
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FIG. 9: Plot of the correlations c(fc) defined in Eq. |S|between 
configurations of record energy in valley nv = k and nv ~ 0. 
As in Fig. |S| the r-dependence here is monotone. The search 
for small r quickly decorrelates, but reaches a plateau value 
for T > 1. 



correlation between c(fc) = c{nv — 0; ny 
Hamming distance H{k) between them: 
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Fig. ini shows that the extremal search decorrelates uni- 
formly faster for decreasing r, similar to a thermal search 
for increasing T. But for r > 1 the correlations decay 
about exponentially with the valley index and appear 
to converge to a plateau value Coq at large k, similar 
to a low-temperature search. Such a plateau indicates 
a certain number of "frozen" variables which provide a 
"backbone" for any near-optimal configuration. In con- 
trast, for T < 1 the extremal search decorrelates almost 
instantly, and there is only a plateau dX H = N/2, the 
most likely distance for any two random configurations. 
Finally, we also look at the hitting time for energy 
records with a valley, which illuminates their internal 
structure. If we call t^ ("waiting time") the time when 
a valley was entered, tm the time when the energy mini- 
mum was hit, and t,j. the time when the valley was exited 
(and the next one entered), then Ref. Q defined a rela- 
tive hitting time as 
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(called "r" in Ref. 5]). For valleys without any struc- 
ture, the average h would be close to zero, but close to 
one for highly structured valleys. In Ref. 5] for the ther- 
mal search at low temperatures it was found that the 
distribution for h is peaked at large h values, indicative 
of the high internal structure of the spin glass valleys. In 
Fig. El we plot the probability density function G{h, t^) 
of observing hitting time /i in a valley entered at i^. We 
find a strong variation with the parameter r in the way t- 
EO explores that internal structure. For large r, the ex- 
tremal search behaves similar to the lowest-temperature 




FIG. 10: Plot of the hitting probability G{h, t^) as a function 
of the hitting time h defined in Eq. JSJ for valleys with increas- 
ing entry times tu,. Shown are the probabilities for r-EO for 
T = 0.2, 1.2, and 2.2, all at L = 16. Note that for r = 1.2 
a near-uniform distribution is quickly reached while for small 
and large r small and large hitting times h predominate, resp. 



thermal search, i.e. G is right-skewed, while for very small 
T, the extremal search seems to ignore the internal struc- 
ture and discovers what it considers the minimum quite 
quickly. For the search at r — 1.2, which typically pro- 
vides the best energy overall, h is almost uniformly dis- 
tributed over the time spent in the valley, at least for 
large i^. 

In Fig.^] we find that the average hitting time {h) de- 
creases for increasing valley index, ever more rapidly for 
smaller r, until it saturates. It may appear that hitting 
onto the record minimum soon after entering a valley may 
be a good thing for an optimization with a local search. 
Yet, since h is measured relative to the length of the res- 
idence within a valley, small {h) here means only that it 
takes a long time to exit a valley, without ever taking full 
account of its internal structure. The best compromise 
in terms of finding energy records (and leaving valleys) 
quickly seems to be provided by r = 1.2. 



B. Varying System Size 



Based on the findings in Sec. lIV Al it appears that the 
most favorable behavior of r-EO from the standpoint of 
optimization is obtained near r = 1.2. To study the 
behavior of the extremal search for varying system size 
in comparison with the thermal search in Ref. 5] , we will 
therefore fix t = 1.2 in this section. 

First, we look again at the number of valleys found 
for increasing runtime. As Fig. 1121 shows, the gain in 
the number of valleys entered increases roughly on a log- 
arithmic timescale. The growth slows at later times, 
apparently due to system size effects, since the increase 
becomes more linear, and differences between data less 
pronounced, for increasing system size. These findings 
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FIG. 11: Plot of the average hitting time {h) defined in Eq. © 
as a function of the valley index nv- For too small r, extremal 
search quickly saturates at short hitting times, unable to leave 
the current valley. There is a noticeable transition in behavior 
for r > 1 with a linear decay in (h) with a slope that slowly 
decreases with increasing r. 
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FIG. 12: Plot of the valleys found by EO on a logarithmic 
timescale at fixed r = 1.2 but for system sizes L — 8, 12, 
16, and 20. Progress toward new valleys occurs linearly on 
a logarithmic time scale, altered at later times by finite size 
effects, which become less pronounced for larger L. 



are very similar to fixed-temperature data (see inset of 
Fig. l)'inRef. ^5]. 

After one system sweep from a random initial state, 
the energy per spin reaches a typical level that strongly 
depends on t, as Fig.2|shows, albeit very little on L, ac- 
cording to Fig. 1131 But progressing further toward lower 
energy records through subsequent valleys soon yields di- 
minishing returns. In contrast, for the thermal search in 
Ref. 5] (see Fig. 4 there) new energy records provide a 
constant gain A in energy, which in itself depends on L 
and T. It should be noted, though, that the extremal 
search reaches extremely low energies quickly and be- 
comes sensitive to the presence of the ground state. The 
behavior of a thermal algorithm is more closely resem- 
bled at smaller r, such as r = 0.7 in Fig.^ for which the 
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FIG. 13: Plot of the energy records found by EO within a 
given valley as a function of the valley index ny- Here r = 
1.2 is fixed and system size L is varied. The dashed line 
at ~ —1.70 marks the average ground state energy density 
for L — > oo |23 |. Convergence toward this limit at our fixed 
runtime becomes ever more difficult for increasing L. Yet, the 
saturation at larger nv clearly indicates the proximity of the 
ground state effecting the search. 



decrease in energy scales linearly with ny- 

As in Fig. O for r > 1, Fig. E| shows that barrier 
heights scale exponentially for r = 1.2, independent of 
system size. In fact, that data for different L appears to 
collapse automatically, without any rescaling. This effect 
can be explained by the definition of barrier heights as 
measured relative to the lowest preceding energy record. 
This would imply that to leave the i-th valley an ex- 
tremal search needs to scale a certain barrier height that 
is largely insensitive to the system size. (Any deviation 
from collapse could well be due to a certain arbitrariness 
in gauging ny = after a single sweep.) Although on 
a different scale, the barrier heights for a thermal search 
were also found to be only weakly dependent on system 
size (see Fig. 2 in Ref. Q). 

Hamming distances between consecutive energy 
records are shown in Fig. 1151 Similar to the barrier 
height, the Hamming distance also grows exponentially 
with the valley index ny, showing a significant, but non- 
extensive, L-dependence. The Hamming distances found 
for T ~ 1.2 here correspond about to the largest found 
in the thermal search in Ref. ^ at the highest tempera- 
tures, i. e. just below Tg. This demonstrates the breadth 
of the extremal search for an optimal choice of r. 

Eliminating the valley-index dependence between bar- 
rier heights in Fig. 1141 and Hamming distances in Fig. 1151 
as in Fig. [7| we again find a nearly linear relation. This 
fact re-affirms the purely geometrical origin of this rela- 
tion, independent of system size. 

Finally, in Fig.^lwe show the behavior of the average 
hitting time (h) for different L. In each case, (h) falls lin- 
early with valley index ny with a size-independent slope. 
Hence, the data can be collapsed with a simple shift in 
nv- That shift is not uniform in L, which could well be 
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FIG. 14: Plot of the barrier height as a function of valley index 
nv as in Fig.|S]but for fixed r = 1.2 and different system sizes 
L. Since r > 1, barrier heights rise exponentially. Barriers 
are measured relative to the most recent low-energy record 
and appear to be independent of L. 
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FIG. 16: Plot of the Hamming distance as a function of the 
barrier height at fixed r = 1.2 and variable L. This data was 
obtained from Figs. I14l and ll5l bv eliminating the valley-index 
dependence between them. Linearity is exemplified by the 
dashed line of slope 1. 
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FIG. 15: Plot of the Hamming distance between successive 
low-energy records as a function of valley index nv at fixed 
r = 1.2 and variable L. The inset shows the same data 
rescaled by N°-^ where N ^ L^ . 
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FIG. 17: Plot of the average hitting time (ft) defined in Eq. Q 
as a function of the valley index nv for fixed r = 1.2 and 
various L. Like in Fig. 1111 {h) falls linearly with nv, but with 
a L-independent slope. The inset shows that a simple shift in 
nv collapses the data for large nv quite well. 



due to the ambiguity in gauging ny- 



V. CONCLUSIONS 

The energy landscape of a spin-glass is probed in this 
work with a non-thermal optimization algorithm, Ex- 
tremal Optimization (EO). Energy values of record mag- 
nitude are used to partition the states visited into a se- 
quence of valleys, indexed by ny. Within each valley 
the state of lowest energy, or minimum, is the best result 
obtained with the algorithm on a given time scale. The 
energy barrier separating two consecutive minima is, by 
construction, the largest ever barrier scaled on the same 
time scale. The sole adjustable parameter of EO, r and 
the system size N = L^ are both varied systematically in 



the investigations. 

Comparing our present results to those of Ref. p, 
which uses the same classification scheme in connection 
with a thermal algorithm, the Waiting Time Algorithm 
(WTM), we are able to disentangle the intrinsic, or ge- 
ometrical, properties of the landscape from those specif- 
ically linked to the two algorithms. These aspects are 
separately discussed in this sequel. 

Both EO and WTM uncover a non-trivial valley struc- 
ture whose overall features are broadly similar. The first 
observation is that, in order to find states of lower en- 
ergy, progressively higher barriers must be surmounted. 
The extremal property of these intervening barriers im- 
plies that the sequence of lowest minima encountered can 
(approximately) be treated as a Poisson process with log- 
arithmic time arguments |5|. Such description can only 



10 



work as long as the system remains far away from the 
global minimum. Indeed, the number ny of valleys ex- 
plored grows on average in near logarithmic fashion and 
the decreasing logarithmic slope apparent for the 'best' 
value of T likely reflects the fact that once the system 
is near to the global optimum, further improvements be- 
come harder to achieve. 

The magnitude of the highest barrier scaled grows 
linearly with ny in the WTM analysis, but exponen- 
tially with EO. The same is true for the Hamming dis- 
tance between consecutive minima. By eliminating ny, 
from these exponential relationships a linear dependence 
emerges, which links the barrier and the Hamming dis- 
tance between consecutive minima. This concurs with 
the WTM result, and the linear dependence thus stands 
out as a geometric property of the energy landscape. For 
growing system size, and r = 1.2, the Hamming dis- 
tance between consecutive minima grows as N'^'^, which 
is qualitatively similar to the N'^-^^ scaling found with 
the WTM. Also similar is the decay of the configuration 
overlap between minima k valleys apart, in both cases 
nearly exponential in k. 

The aspect where WTM and EO mostly differ is in 
the distribution of the time spent searching for the low- 
est energy state within a fixed valley. Unlike the WTM, 
EO locates this minimum fairly quickly, at least for val- 
ues of T close to 1.2, which is the best value in terms of 
optimization performance. In other words, EO seems to 
be able to roam more easily through configuration space, 
which is in accord with the original intention behind its 



design. Short-ranged spin glasses are probably the test 
problem where EO is at its worst in terms of optimiza- 
tion performance, yet its performance is slightly better 
than what WTM can achieve in terms of, say, the lowest 
energy achieved in a fixed number of updates. However, 
from a broader optimization perspective the performance 
of the two algorithms is similar, since lower energy value 
are logarithmically spaced in time. This, we suggest, 
might be a general feature of local searches in complex 
landscapes with a large number of near-equivalent min- 
ima. 

From a physics point of view, applying a non-thermal 
algorithm to landscape exploration removes the appeal of 
using concepts adapted from thermal-equilibrium, such 
as the free energy, to describe the dynamics. Since, as 
we have shown, the process of jumping from one valley to 
a lower-lying valley, is only weakly dependent on the algo- 
rithm chosen, thermal concepts are likely to be generally 
irrelevant for the drift part of the dynamics even though 
the dynamical update rules obey detailed balance. As we 
have argued elsewhere jSfl |31| the reason for this is that 
these jumps are effectively irreversible on the time scale 
at which they occur. 
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